Special sequential estimation problems in Markov processes

The KTH Visit in Semi-Markov Processes

Empirical Bayes Estimation in Nonstationary Markov chains

Estimation procedures for nonstationary Markov chains appear to be relatively sparse. This work introduces empirical  Bayes estimators  for the transition probability  matrix of a finite nonstationary  Markov chain. The data are assumed to be of  a panel study type in which each data set consists of a sequence of observations on N>=2 independent and identically dis...

Order estimation for a special class of hidden Markov sources and binary renewal processes

We consider the estimation of the order, i.e., the number of hidden states, of a special class of discrete-time finite-alphabet hidden Markov sources. This class can be characterized in terms of equivalent renewal processes. No a priori bound is assumed on the maximum permissible order. An order estimator based on renewal types is constructed, and is shown to be strongly consistent by computing...

Sequential Testing Problems for Poisson Processes

We present the explicit solution of the Bayesian problem of sequential testing of two simple hypotheses about the intensity of an observed Poisson process. The method of proof consists of reducing the initial problem to a free-boundary differential-difference Stephan problem, and solving the latter by use of the principles of smooth and continuous fit. A rigorous proof of the optimality of the ...

Optimal Stopping Problems for Some Markov Processes

In this paper, we solve explicitly the optimal stopping problem with random discounting and an additive functional as cost of observations for a regular linear diffusion. We also extend the results to the class of one-sided regular Feller processes. This generalizes the result of Beibel and Lerche [Statist. Sinica 7 (1997) 93–108] and [Teor. Veroyatn. Primen. 45 (2000) 657–669] and Irles and Pa...